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I.  INTRODUCTION 


Lowell  Bruce  Anderson  and  Jerome  Bracken 

The  purpose  of  this  paper  is  to  discuss  some  aspects  of  the 
use  of  MaxMin  and  MlnMax  strategies  in  the  analysis  of  multi- 
stage games.  This  discussion  is  motivated  by  the  development  of 
the  ACDA  Tactical  Air  Campaign  Model  (ATACM),  which  is  documented 
in  References  [9]  and  [10].  ATACM  proposes  the  use  of  approxi- 
mate MaxMin  and  MinMax  strategies  Instead  of  optimal  mixed 
(equilibrium)  strategies.  Chapter  II,  below,  gives  a critique 
of  the  MaxMin,  MinMax  approach  as  implemented  in  ATACM.  Chapter 
III  treats  several  computational  aspects  of  true  MaxMin  and  Min- 
Max strategies;  and  Chapter  IV  gives  a result  concerning  MaxMin 
and  MinMax  values  in  adaptive  and  nonadaptlve  games.  A more 
detailed  summary  of  these  chapters  follows. 

Chapter  II  is  a rather  comprehensive  critique  of  ATACM. 

The  principal  criticisms  go  to  the  heart  of  the  MaxMin,  MinMax 
approach  as  it  is  implemented  in  ATACM.  First,  if  one  player 
plays  a conservative  strategy,  the  other  player  upon  observing 
this  can  drive  the  outcome  down  towards  the  payoff  corresponding 
to  that  conservative  strategy.  Optimal  mixed  strategies,  on  the 
other  hand,  yield  expected  results  that  cannot  be  driven  down 
towards  a conservative  payoff.  Second,  conservative  strategies 
may  give  results  that  are  insensitive  to  important  force  struc- 
ture changes,  while  optimal  mixed  strategies  could  properly 
reflect  the  importance  of  these  changes.  And  third,  if  the 
game  being  modeled  is  inherently  stochastic  (which  air  combat 
is),  then  MaxMin  and  MinMax  strategies  also  yield  only  expected 
results,  not  guaranteed  bounds  as  claimed  in  ATACM.  Two  other 
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aspects  of  ATACM  are  also  criticized:  (1)  the  ATACM  approxima- 

tion procedure,  and  (2)  the  ATACM  assessment  procedure.  Finally, 
some  suggestions  for  improvement  are  made.^ 

If  the  optimal  mixed  strategies  and  game  value  were  known, 
then  knowing  the  true  MaxMin  and  MinMax  strategies  could  be  use- 
ful additional  Information.  The  OPTSA  models  (References  [3], 
[^],  and  [5])  calculate  the  optimal  strategies  and  game  values 
for  the  games  they  address;  but,  as  currently  programmed,  they 
cannot  calculate  the  MaxMin  or  MinMax  strategies.  Chapter  III 
treats  several  aspects  of  the  computation  of  MaxMin  and  MinMax 
strategies  in  multi-stage  games.  A new  method  for  finding  Max- 
Min and  MinMax  strategies  for  one-stage  games  is  proposed.  Com- 
putation of  exact  MaxMin  and  MinMax  strategies  for  multi-stage 
games  is  discussed,  and  computation  times  are  estimated. 

MaxMin  and  MinMax  strategies  can  be  considered  for  several 
types  of  games,  two  of  which  are:  (a)  nonadaptive  games,  and 

(b)  behavioral  games,  which  in  Reference  [13]  are  shown  to  be 
equivalent  to  adaptive  games.  The  relationship  between  the  Max- 
Min and  MinMax  strategies  of  an  adaptive  game  and  the  MaxMin  and 
MinMax  strategies  of  the  corresponding  nonadaptive  game  is  dis- 
cussed in  Chapter  IV. 


'it  is  appropriate  to  remark  that  ATACM  shares  with  DYGAM  (References  [6] 
and  [7])  the  characteristic  that  the  stated  problem  can  be  solved  rigor- 
ously except  for  approximation  error.  Counterexanples  for  other  models 
not  having  this  guaranteed  optimization  philosophy  are  provided  in 
References  [8]  and  [11]. 
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II.  A CRITIQUE  OF  ATACM 
Lowell  Bruce  Anderson 


The  ACDA  Tactical  Air  Campaign  Model  (ATACM)  is  described 
in  References  [9]  and  [10].  Reference  [9]  claims  that  the 
specific  features  of  ATACM  are  that  ATACM  permits 

(1)  as  many  as  four  user-defined  aircraft  types  per 
side  and  as  many  as  eight  different  missions  per 
aircraft  type; 

(2)  automatic  generation  of  approximate,  optimal 
enforceable  aircraft  allocation  strategies  as 

a function  of  stage  for  any  subset  of  the  missions 
for  which  user-specified  fractions  are  not  supplied; 

(3)  calculation  of  firm  upper  and  lower  bounds  on  the 
objective  function  value  associated  with  the  en- 
forceable strategies  employed; 

(^)  the  option  to  use  a weighted  sum  of  three  different 
objective  functions  as  the  criterion  for  generating 
the  optimal  strategies; 

(5)  the  option  to  individually  weight  the  Blue  and  Red 
contributions  to  these  objective  functions  as  a 
function  of  stage;  and 

(6)  the  option  to  specify  fractional  or  numerical 
reinforcements  for  any  aircraft  type  as  a func- 
tion of  stage. 

If  the  procedures  used  in  ATACM  provide  useful  information, 
then  feature  (1)  above  would  be  very  significant  and  important, 
and  would  make  ATACM  the  premier  model  in  its  field.  On  the 
other  hand,  feature  (1)  is  largely  irrelevant  if  ATACM  does 
not  provide  useful  Information.  Whether  ATACM  provides  use- 
ful information  or  not  depends  on  what  one  means  by  useful 
information,  which  in  turn  largely  depends  on  the  definitions 
and  interpretations  associated  with  features  (2)  and  (3)>  and 
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on  the  acceptability  of  the  assessment  methodology.  In  this 
paper,  we  concentrate  on  the  definitions  and  assumptions 
associated  with  features  (2)  and  (3)  because  the  assessment 
methodology  of  ATACM  could  be  changed  if  warranted.* 

The  key  terms  in  feature  (2)  are  "approximate,  optimal, 
enforceable  ...  strategies."  These  terms  are  not  directly 
defined  in  Reference  [9].  However,  it  is  clear  from  the  de- 
tails of  References  [9]  and  [10]  that  by  "optimal  enforceable 
strategies,"  the  developers  of  ATACM  mean  MaxMin  and  MinMax 
strategies.  This  distinction  is  Important  because  the  stan- 
dard definition  of  "optimal  strategies"  for  a two-person  zero- 
sum  game  (like  the  game  in  ATACM)  is  that  optimal  strategies 
are  the  (possibly  mixed)  equilibrium  strategies.^  Thus,  the 
claim  of  feature  (2)  is,  at  best,  misleading.  Properly  phrased, 
feature  (2)  should  be  stated  as: 

(2')  ATACM  generates  strategies  that  are,  in  some 

sense,  approximations  to  the  MaxMin  and  MinMax 
strategies  of  the  game  played  in  ATACM. 

This  revised  statement  of  feature  (2)  raises  two  questions: 

How  worthwhile  is  it  to  generate  MaxMin  and  MinMax  strategies 
in  lieu  of  optimal  (equilibrium)  strategies,  which  ATACM 
cannot  generate?  How  good  are  ATACM 's  approximations  to  the 
MaxMin  and  MinMax  strategies?  The  second  question  is  related 
to  feature  (3),  which  claims  that  ATACM  calculates  firm  upper 
and  lower  bounds  on  (properly  phrased)  the  payoffs  produced 
by  the  MaxMin  and  MinMax  strategies.  By  "firm  bounds"  the 
developers  of  ATACM  apparently  mean  true  bounds,  not  tight 


*A  few  specific  changes  to  the  assessment  methodology  will  be  suggested  below. 
Of  course,  a sufficient  amount  of  changes  in  the  assessment  methodology 
could  increase  the  ccxiputer  running  time  to  the  point  where  it  is  no  longer 
practical  to  use  ATACM.  Thus,  the  blanket  statement  above  that  "the  assess- 
ment methodology  could  be  changed"  is,  in  general,  an  over sliiplificat ion. 

^Accordingly,  throughout  this  chapter  we  will  use  the  term  "optimal  strate- 
gies" to  mean  the  (possibly  mixed)  equilibrium  strategies. 


bounds.  But  it  is  trivial  to  calculate  true  bounds  if  one  does 
not  care  how  tight  these  bounds  are,  and  true  but  very  loose 
bounds  can  be  quite  useless. 

In  Section  A we  will  discuss  the  limitations  of  consider- 
ing only  MaxMin  and  MinMax  strategies,  as  is  done  in  ATACM.  In 
Section  B we  will  discuss  the  approximation  procedures  used  in 
ATACM.  In  Section  C we  will  discuss  some  limitations  of  the 
assessment  procedure  used  in  ATACM.  Finally,  in  Section  D we 
will  make  a suggestion  that  might  make  ATACM  a more  useful 
model,  provided  that  this  suggestion  can  be  Implemented  without 
greatly  increasing  the  computer  running  time. 

A.  SOME  ADVANTAGES  AND  LIMITATIONS  OF  MAXMIN  AND  MINMAX 

MaxMin,  MinMax,  and  optimal  (equilibrium)  strategies  are 
equivalent  for  any  two-person,  zero-sum  game  with  a saddle- 
point.  So  for  this  section  (only)  suppose  that  the  game  under 
consideration  does  not  have  a saddlepoint. 

1 . A Discussion  of  the  Claimed  Advantages  of  MaxMin  and  MinMax 

Two  advantages  of  considering  MaxMin  and  MinMax  strategies, 
as  opposed  to  optimal  mixed  strategies,  are  claimed  in  Reference 
[10].  These  two  advantages  are  summarized  as  follows:  (1)  Max- 

Min and  MinMax  strategies  are  pure  strategies,  and  many  military 
commanders  might  abhor  the  concept  of  randomization  to  decide 
each  day's  aircraft  assignment.  (2)  The  "game"  of  a war  in 
Europe  will  be  "played"  once  at  most.  Optimal  mixed  strategies 
guarantee  to  each  side  only  that  the  side's  expected  payoff  will 
not  be  less  than  a specific  amount  (the  value  of  the  game).  The 
actual  payoff  to  either  side  is  a random  variable  which  may  be 
above  or  below  the  expected  payoff.  On  the  other  hand,  conser- 
vative play  (MaxMin  or  MinMax,  as  appropriate)  will  guarantee 
to  each  side  an  actual  payoff  that  is  greater  than  or  equal  to 
the  worst  the  side  could  receive  with  optimal  mixed  strategies 
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although  not  as  good  as  the  expected  payoff  from  optimal  mixed 
strategies. 

We  believe  that  these  claimed  advantages  are  not  as  great 
as  they  first  might  appear.  First,  while  commanders  might  not 
flip  a coin  to  decide  how  to  allocate  their  aircraft,  they  would 
attempt  to  avoid  making  decisions  in  a completely  predictable 
manner.  Indeed,  they  would  attempt  to  exploit  an  enemy's  pre- 
dictability and  they  might  even  attempt  to  set  up  and  fake  out 
an  enemy. * Playing  optimal  mixed  strategies  is  not  a perfect 
way  to  model  each  side's  attempt  to  exploit  his  enemy's  predic- 
tability and  surprise  him  when  appropriate.  However,  playing 
mixed  strategies  seems  to  us  to  be  a better  way  to  reflect  these 
characteristics  of  war  rather  than  playing  that  each  side  uses 
his  conservative  pure  strategy  throughout  the  war.  Thus,  if 
there  is  a significant  difference  between  the  MaxMin  value  and 
the  MinMax  value  (so  that  there  is  much  to  be  gained  by  sur- 
prise), playing  optimal  mixed  strategies  may  well  be  more 
realistic,  not  less  realistic,  than  playing  conservative  MaxMin 
and  MinMax  strategies. 

The  second  argument  above,  that  MaxMin  and  MinMax  strate- 
gies are  more  appropriate  than  optimal  mixed  strategies  for  a 
game  (or  war)  that  will  be  played  (or  fought)  only  once,  is  a 
long-standing  point  of  discussion  in  game  theory.  If  the  pay- 
offs to  each  player  satisfy  typical  axioms  for  utilities  (such 
as  in  von  Neumann  and  Morgenstern,  Reference  [18],  or  as  in 
Luce  and  Raiffa,  Reference  [15]) > then  the  situation  is  clear: 
optimal  mixed  strategies  are  more  appropriate  than  MaxMin  and 
MinMax  strategies.  For  example,  if  the  payoffs  in  the  example 
of  Reference  [10]  are  in  terms  of  utilities  to  Blue,  then  Blue 
is  Indifferent  between  an  expected  payoff  of  3-^  and  a certain 
payoff  of  and  he  prefers  either  to  a certain  payoff  of  2.0. 


* "Surprise"  is  a principle  of  war,  "conservative  play"  is  not. 
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The  problem  is  that  it  Is  relatively  much  easier  to  model  a 
physical  occurrence  such  as  Blue  minus  Red  firepower  delivered, 
than  it  is  to  determine  Blue's  and  Red's  utilities  for  deliver- 
ing firepower. 

What  one  should  do  if  it  is  not  known  whether  the  payoffs 
satisfy  the  axiomatic  conditions  of  utilities  is  not  clear,  and 
we  will  certainly  not  resolve  that  issue  here.  But  there  is  an 
intuitive  belief  that  if  a game  is  played  many  times  and  no  one 
play  of  the  game  strongly  affects  the  end  result,  then  the  con- 
ditions of  utility  are  "approximately  satisfied"  and  the 
expected  payoff  is  a reasonable  measure.  (The  developers  of 
ATACM  agree  with  this  intuition  in  Reference  [10].)  On  the 
other  hand,  if  the  game  is  played  only  once  and  there  are  signi- 
ficant differences  in  possible  outcomes,  then  the  conditions  of 
utility  might  not  be  satisfied.^ 

While  the  war  will  be  fought  (at  most)  once,  aircraft  allo- 
cation decisions  will  not  be  made  only  once.  A commander  could 
decide  to  re-allocate  his  aircraft  for  each  raid  on  each  day  of 
the  war,  and  he  could  make  one  allocation  in  one  part  of  the 
theater  and  another  in  another  part.  For  example,  a 30-day 
war  with  three  raids  per  day  into  two  areas  of  the  theater 
could  result  in  l80  allocations.  Thus,  as  in  many  plays  of  one 
game,  the  commander  has  many  distinct  allocation  decisions.  So 
in  this  sense,  the  sequential  game  of  aircraft  allocation  is 
intuitively  similar  to  many  plays  of  one  game  and  expected  pay- 
off would  be  the  preferred  measure.  On  the  other  hand,  it  is 
possible  that  an  "unlucky"  decision  on  the  first  raid  of  the 
first  day  could  have  a dominant  Impact  on  the  rest  of  the  war. 


'it  should  be  noted  that  this  structure  is  intuitive,  not  fonnal,  because 
a game  played  many  times  can  be  thought  of  as  one  l^ge  game  with  many 
sub-games  inside  of  it,  and  the  large  game  is  then  played  only  once. 

But  given  this  intuitive  structure  (as  opposed  to  a formal  structure), 
one  can  ask  how  the  sequential  game  of  aircraft  allocations  fits  into 
the  Intuit  lai. 
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no  matter  what  decisions  are  made  for  the  rest  of  the  war.  To 
the  extent  that  this  dominance  can  occur,  a sequential  game  is 
intuitively  similar  to  one  play  of  one  game.  Considering  only 
this  argument,  expected  payoff  may  not  be  clearly  preferred 
over  MaxMin  and  MinMax  as  a measure  of  effectiveness,  but  it 
is  not  clearly  inferior  either.  Accordingly,  the  validity  of 
the  second  claimed  advantage  of  MaxMin  and  MinMax  over  optimal 
mixed  strategies  is  also  in  doubt. 

Another  facet  of  the  second  claimed  advantage  of  the  Max- 
Min, MinMax  approach  is  that,  while  the  assessment  portion  of 
ATACM  is  deterministic,  actual  combat  is  not  deterministic. 

Thus,  even  if  the  entries  in  the  payoff  matrix  are  truly  the 
expected  outcomes  of  an  air  war,  the  MaxMin  of  these  entries 
is  the  MaxMin  of  expected  results,  and  the  MinMax  is  the  Min- 
Max of  expected  results.  Accordingly,  whether  a commander 
plays  a MaxMin  strategy  or  a mixed  strategy,  all  he  can  count 
on  is  an  expected  result  of  non-determlnistic  combat,  not  a 
certainty.  Either  way,  to  quote  from  Reference  [10],  "...the 
outcome  is  enforceable  by  the  two  sides  only  in  an  expected 
value  sense,"  and  so  this  claimed  advantage  of  MaxMin  and  Min- 
Max over  optimal  mixed  strategies  is  really  no  advantage  at  all. 

2 . Some  Limitations  of  MaxMin  and  MinMax 

The  discussion  above  attempts  to  counter  the  two  claimed 
advantages  for  considering  MaxMin  and  MinMax  strategies  in 
lieu  of  optimal  mixed  strategies.  Combining  these  counter 
arguments  gives  a major  limitation  of  using  MaxMin  and  MinMax 
strategies:  A side  will  try  to  exploit  its  enemy's  predicta- 

bility if  it  has  the  opportunity  and  there  is  a payoff  from 
doing  so.  If  one  side  sticks  to  its  conservative  play  MaxMin 
strategies,  then  the  other  side  could  observe  this  over  the 
course  of  the  war  and  allocate  its  aircraft  specifically  against 
that  MaxMin  strategy  (instead  of  using  its  own  MinMax  strategy). 
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This  would  push  the  payoff  to  the  MaxMin  side  down  toward  the 
MaxMin  value. 


The  hypothetical  example  of  Reference  [10]  is  given  below; 
Red 


CAS 

ABA 

INT 

.8 

.2 

0 

CAS 

4 

1 

6 

.4 

4 

1 

6 

1 

Blue  ABA 

3 

5 

2 

.6 

3 

5 

2 

2 

INT 

2 

0 

2 

0 

2 

0 

2 

0 

MaxMin  = 2,  MinMax  = , expected  value  = 3.^ 


If  Blue  plays  his  MaxMin  strategy  (ABA)  on  each  day  of  the  war, 
then  Red  can  observe  this  and  fly  INT  instead  of  CAS.  This 
strategy  may  push  the  payoff  to  Blue  towards  2.0.  If  so.  Blue 
will  have  to  either  accept  a payoff  nearer  2.0  or  fly  CAS 
instead.  How  close  the  payoff  is  to  2.0  depends  on  when  Red 
starts  flying  INT  and  on  the  details  of  the  assessment  proce- 
dure. But  it  is  not  at  all  clear  that  Red  would  see  Blue  fly- 
ing ABA  each  day  and  yet  never  fly  INT.  And  it  is  not  at  all 
clear  that  if  Red  started  flying  INT,  Blue  would  still  continue 
to  fly  ABA  (in  the  example  game). 

In  summary,  given  that  each  side  must  make  a decision  day 
after  day,  it  is  not  reasonable  that  one  side  would  see  the 
other  play  its  conservative  strategy  and  not  adjust  its  own 
strategy  accordingly  to  drive  the  payoff  toward  the  MaxMin  (or 
MinMax)  value.  Playing  mixed  strategies  may  be  a reasonable 
way  to  model  the  case  where  both  sides  continually  adjust  their 
strategies  to  anticipate  the  moves  of  the  other. 

A second  limitation  of  considering  MaxMin  and  MinMax  stra- 
tegies is  that,  since  these  are  pure  strategies,  they  may  be 
insensitive  to  important  force  structure  changes.  For  example. 
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suppose  that  two  force  structures  are  under  consideration  and 
for  both  force  structures  the  Blue  MaxMin  strategy  is  to  fly 
all  ABA  while  the  optimal  mixed  strategy  is  to  fly  both  CAS 
and  ABA.  Suppose  further  that  the  second  force  structure 
results  in  a slightly  lower  payoff  from  flying  ABA  but  a much 
greater  payoff  from  flying  CAS.  Then  the  MaxMln  structure 
would  prefer  the  first  force  structure  over  the  second.  A 
numerical  example,  following  Reference  [10],  is  as  follows: 

BLUE  FORCE  STRUCTURE  NO.  1 


CAS 

ABA 

INT 

CAS 

H 

1 

6 

Blue  ABA 

3 

5 

2 

INT 

2 

0 

2 

5 

6 

MaxMin  = 2,  MinMax  = MaxMin  vs.  MlnMax  Payoff  = 3 
Optimal  mixed  payoff  = 3-^ 

BLUE  FORCE  STRUCTURE  NO.  2 


CAS 

ABA 

INT 

CAS 

^100 

0 

600 

Blue  ABA 

2 

500 

1 

INT 

1 

0 

1 

^100 

500 

600 

MaxMin  = 1,  MlnMax  = ^00,  MaxMin  vs.  MinMax  Payoff  = 2 
Optimal  mixed  payoff  > 200 

In  the  above  example,  force  structure  1 has  both  a better  MaxMin 
and  a better  MaxMin  versus  MinMax  payoff  to  Blue  than  force 
structure  2 does,  yet  force  structure  2 would  appear  to  be  much 
better  for  Blue. 
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The  numerics  of  the  above  example  are  not  Important.  What 
is  important  is  that  a MaxMin,  MinMax  approach  can  overlook  the 
capability  of  general  purpose  aircraft  to  fly  any  one  of  several 
missions  without  the  enemy  knowing  in  advance  which  mission  will 
be  flown.  Accordingly,  the  MaxMin,  MinMax  approach  can  give  an 
unrealistic  advantage  to  a special  purpose  aircraft  that  might 
be  only  slightly  better  on  one  mission  and  much  worse  on  all 
other  missions  than  an  alternative  general  purpose  aircraft.  It 
may  even  be  possible  that  if  a special  purpose  aircraft  is  bought 
by  the  MaxMin  side  in  place  of  a general  purpose  aircraft,  then 
that  side's  enemy  might  more  easily  force  the  outcome  of  the  war 
down  towards  the  MaxMin  value. 

Finally,  there  is  the  problem  of  how  one  uses  the  MaxMin 
and  MinMax  strategies  and  values.  If  an  analyst  is  comparing 
two  force  structures,  he  might  prefer  a force  with  much  higher 
MaxMin  value  and  a slightly  lower  game  value  when  compared  with 
an  alternative  force.  However,  ATACM  does  not  permit  such  a 
comparison  because  it  cannot  compute  the  optimal  (mixed)  stra- 
tegies or  the  game  value.  Instead,  the  developers  of  ATACM 
seem  to  suggest  considering  the  value  procedure  by  playing  the 
MaxMin  strategy  versus  the  MinMax  strategy  (conservative  play 
on  both  sides).  But  this  "conservative  play  payoff"  does  not 
depend  on  any  of  the  possible  payoffs  of  the  game  (except  for 
itself),  other  than  that  it  must  be  above  the  MaxMin  payoff 
and  below  the  MinMax  payoff.  That  is,  changing  one  entry  in  the 
(complete)  game  payoff  matrix  can  make  this  payoff  as  high  as 
the  MinMax  or  as  low  as  the  MaxMin.  It  seems  to  us  that  this 
"conservative  play  payoff"  is  an  arbitrary  number  and  the  only 
Justification  for  considering  this  payoff  as  a measure  is  the 
one  implied  by  the  developers  of  ATACM:  that  the  two  commanders 

would  actually  use  MaxMin  and  MinMax  strategies.  We  believe 
this  to  be  a weak  argument  for  the  reasons  given  above. 
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B.  LIMITATIONS  OF  THE  ATACM  APPROXIMATION  PROCESS 


ATACM  makes  two  approximations  in  calculating  the  MaxMln 
and  MinMax  values,  and  it  gives  the  strategies  that  produce 
these  approximations.  Three  points  concerning  these  approxi- 
mations are  important. 

First,  these  two  approximations  are  made  whether  or  not 
the  game  has  a saddlepolnt  (and  so  the  discussion  below  applies 
whether  or  not  the  game  has  a saddlepolnt). 

Second,  the  approximate  values  are  not  necessarily  close 
to  the  true  MaxMin  or  MinMax  values.  Let  m and  M denote  the 
true  MaxMln  and  MinMax  values,  respectively,  of  the  game;  and 
let  in  and  M denote  the  ATACM  approximations  to  these  values. 

It  is  true  that 

m < m £ M £ M . 

Thus,  if  iii  is  close  (in  some  sense)  to  M,  then  not  only  are  the 
approximations  of  m to  m and  M to  M good,  but  m and  M are  close 
to  each  other  and  so  are  necessarily  close  to  the  game  value 
(produced  by  the  optimal,  possibly  mixed,  strategies)  which 
lies  between  them.  Thus  if,  for  a particular  case,  m is  close 
to  M,  then  all  the  arguments  (pro  and  con)  dealing  with  MaxMln 
and  MinMax  in  Section  A are  Irrelevant,  and  ATACM  produces  use- 
ful information  for  that  case.  However,  if  iii  is  not  close  to  M, 
then  ATACM  cannot  indicate  the  relative  closeness  of  m,  m,  M, 
and  M.  It  may  be  that  m is  close  to  m and  M is  close  to  M,  but 
m is  not  close  to  M;  or  it  may  be  that  m = M and  the  approxima- 
tions are  terrible;  or  it  may  be  that  none  of  the  quantities 
are  close  to  any  other. 

Third,  the  strategies  produced  by  ATACM  are  not  the  MaxMin 
and  MinMax  strategies.  The  strategies  produced  are  those  used 
to  obtain  iii  and  M.  In  general,  the  relationship  between  those 
strategies  and  the  true  MaxMin  and  MinMax  strategies  is  not 
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clear.  Thus,  when  m is  not  close  to  M,  m may  not  be  close  to 
m,  and  the  ATACM  "approximate  MaxMin"  strategy  may  not  be  simi- 
lar to  the  true  MaxMln  strategy  (the  same  holds  for  the  MinMax 
strategies) . 

The  two  approximations  made  in  ATACM  are  discussed  on 
pages  30  through  3^  of  Reference  [9]-  We  have  no  rigorous 
basis  for  commenting  on  the  first  approximation,  using  linear 
interpolation  to  generate  a first  pass  approximated  strategy, 
but  intuitively  this  seems  like  a reasonable  approximation. 
However,  as  Reference  [9]  states,  a second  approximation  is 
required  to  obtain  the  strategies  that  yield  m and  M.  This 
second  approximation  is  where  significant  inaccuracies  can 
occur. 

If  Blue  is  the  MaxMin  player,  the  second  approximation 
requires,  for  the  computation  of  the  "approximate  MaxMin" 
strategy,  that  at  several  stages  Blue  resources  be  rounded 
down  to  the  nearest  grid  points  and  that  Red's  resources  be 
rounded  up  to  the  nearest  grid  points.  (The  grid  points  form 
a grid  over  Blue  and  Red  inventories,  and  the  location  of  the 
grid  points  are  input  to  ATACM.)  In  the  example  given  in 
Appendix  A of  Reference  [9],  Blue  has  two  types  of  aircraft. 

Red  has  one  type;  the  grid  points  are  at  0,  333>  667,  and  1000 

for  Blue  plane  type  1 and  at  0,  200,  and  ^100  for  Blue  plane 

type  2 — Red's  grid  points  are  at  0,  ^00,  800,  and  1200  for  its 
plane  type  1.  Thus,  In  the  extreme  case,  if  ATACM  were  com- 
puting the  MinMax  value  and  Blue  had  332  aircraft  of  type  1 
and  199  aircraft  of  type  2 while  Red  had  ^<01  aircraft  of 

type  1,  ATACM  would  round  this  to  no  Blue  aircraft  of  either 

type  and  to  800  Red  aircraft.  We  suspect  that  this  type  of 
approximation  would  generally  lead  to  very  poor  bounds.  In 
Section  D we  suggest  an  improvement  to  this  type  of  approxi- 
mation. 

Results  of  three  ATACM  runs  are  given  on  pages  A-33  through 
A-i)l  of  Appendix  A,  Reference  [9]«  Our  interpretation  of  these 
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runs  may  be  in  error,  but  It  appears  as  if  the  three  runs  were 
made  with  the  same  inputs  except  for  the  initial  number  of  Blue 
and  Red  aircraft.  The  measure  of  effectiveness  used  is  cumula- 
tive Blue  air  plus  ground  firepower  minus  cumulative  Red  air 
plus  ground  firepower  for  a two-day  (stage)  war.^  The  results 
are : 


The  units  of  M and  m are  Important,  and  they  appear  to  be 
closer  to  tons  rather  than  ounces  of  firepower.  For  example, 
in  the  ATACM  data  the  aircraft  of  type  1 deliver  2 units  of 
CAS  firepower  per  sortie  and  they  have  a sortie  rate  of  2.0 
on  CAS  mission.  Thus  it  would  take  over  2,531  Blue  aircraft 
of  type  1 to  deliver  20,250  units  of  firepower  in  two  days, 
assuming  no  attrition  whatsoever.  (But  for  Trial  1,  there  are 
only  iJOO  Blue  aircraft  of  type  1 in  the  theater  on  day  1,  with 
100  replacements  on  day  2,  and  Blue  aircraft  of  type  2 cannot 
deliver  CAS  firepower.)  Similarly,  it  would  take  over  1,687 
Red  aircraft  to  deliver  20,250  units  of  firepower  in  two  days 
with  the  data  in  ATACM,  assuming  no  attrition.  (But  for 
Trial  1,  there  are  only  500  Red  aircraft  in  the  theater  on 
day  1 with  200  replacements  on  day  2.)^ 

Based  on  the  arguments  above,  we  believe  that  M - m is 
a reasonable  measure  of  whether  or  not  ATACM  produces  useful 


^Blue's  air  plus  ground  firepower  is  weighted  by  1/2  on  day  1 and  the  value 
of  residual  Blue  and  Red  aircraft  of  type  1 is  8 units  of  firepower  (Blue 
aircraft  of  type  2 have  no  residual  value). 

^With  bounds  this  wide,  it  seems  to  us  that  no  meaning  whatsoever  can  be 
attached  to  the  value  produced  by  the  "approximate  MaxMin"  strategy 
versus  the  "approximate  MinMax"  strategy. 


Trial 

1 

2 

3 


Red  MinMax  minus  Blue  MaxMin 
(M  - m) 


20,250 

11,042 

16,072 


information.  Clearly  the  value  of  M - m depends  on  the  data 
used  as  well  as  on  the  model,  but  for  the  sample  data  given 
in  Appendix  A of  Reference  [9]  it  appears  that  ATACM  does  not 
produce  useful  information. 

C.  LIMITATIONS  OF  THE  ATACM  ASSESSMENT  PROCESS 

It  should  be  noted  that  developers  of  ATACM  did  not  intend 
to  make  major  contributions  to  the  (assessment)  modeling  of  air 
combat.  For  example,  page  12  of  Reference  [9]  states:  "In  con- 

sideration of  these  limitations,  ATACM  is  purposely  structured 
so  that  other,  alternative  assessment  methodologies  can  be  im- 
plemented with  minimal  programming  effort."  ATACM  uses  features 
from  other  models  in  order  to  build  an  assessment  methodology 
on  which  the  optimization  methodology  could  operate.  Accord- 
ingly, there  are  no  significant  new  features  in  the  assessment 
methodology  of  ATACM.  As  mentioned  in  our  introduction,  we  have 
not  thoroughly  reviewed  the  assessment  methodology  of  ATACM,  and 
so  the  limitations  and  suggestions  for  improvement  given  below 
are  not  necessarily  complete;  however,  they  should  be  relatively 
easy  to  Implement  in  the  ATACM  computer  program. 

1 . Engagements 

It  may  be  that  no  one  ordered  set  of  engagements  for  air 
combat  is  absolutely  correct,  and  it  may  be  that  several  differ- 
ent orderings  of  different  engagements  are  reasonably  acceptable. 
However,  the  particular  set  of  engagements  played  in  ATACM  has 
three  logical  deficiencies  which  might  render  it  unreasonable. 
These  deficiencies  are  that — 

(1)  Air  Base  Defense  (ABD)  aircraft  cannot  engage  enemy 
Rear  (i.e.,  air  base)  SAM  Suppressor  (RSS)  aircraft, 
and  Battlefield  Defense  (BD)  aircraft  cannot  engage 
either  enemy  RSS  aircraft  or  enemy  Forward  (i.e., 
battlefield)  SAM  Suppressor  (FSS)  aircraft.  ABD 
aircraft  should  be  able  to  engage  enemy  RSS  aircraft 
and  BD  aircraft  should  be  able  to  engage  both  enemy 
RSS  and  enemy  FSS  aircraft. 
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(2)  BD  aircraft  cannot  engage  enemy  ABA  aircraft.  Perhaps 
there  is  some  way  for  ABA  aircraft  to  reach  enemy 

air  bases  by  not  going  over  the  FEBA;  but  this  should 
be  an  option  instead  of  a fixed  rule  in  the  model,  and 
if  the  ABA  aircraft  are  not  vulnerable  to  enemy  BD  air- 
craft, then  it  is  hard  to  see  why  ATACM  assumes  they 
are  vulnerable  to  enemy  forward  (battlefield)  SAMs. 

BD  aircraft  should  be  able  to  engage  enemy  ABA  air- 
craft, and  likewise  ABAE  (ABA  Escort)  aircraft  should 
be  able  to  engage  enemy  BD  aircraft. 

(3)  In  ATACM,  Blue  forward  SAMs  shoot  at  Red  CAS  aircraft, 
then  Blue  BD  aircraft  shoot  at  Red  CAS  aircraft , then 
Red  CAS  aircraft  deliver  ordnance  on  ground  units. 

But  if  the  Blue  forward  SAMs  are  near  (or  behind)  the 
ground  units  they  are  protecting,  it  is  hard  to  see 
how  air-to-air  engagements  between  CAS  and  BD  aircraft 
could  occur  between  the  time  that  the  SAMs  shoot  at 
the  CAS  aircraft  and  the  time  that  the  CAS  aircraft 
shoot  back  at  the  units  the  SAMs  are  defending.  This 
order  and  the  corresponding  order  for  ABA  and  ABD 
aircraft  and  rear  SAMs  should  be  changed. 


2 . Attrition  Equations 

Our  main  comment  on  the  equations  in  ATACM  is  that  these 
equations  have  been  pieced  together  out  of  various  other  air 
combat  models  and  no  justification  is  given  in  Reference  [9] 
as  to  why  these  equations,  and  not  others,  were  selected  for 
use  in  ATACM.  Page  l6  of  Reference  [10]  gives  one  sentence  of 
justification,  namely:  ’’Initially  we  propose  using  the  more 

detailed  VECTOR  equations  for  most  Interactions  to  be  modeled.” 
But  it  is  not  clear  in  Reference  [10]  what  is  meant  by  ’’detail,” 
why  detail  is  desired,  which  equations  the  VECTOR  equations  are 
more  detailed  than,  which  interactions  the  VECTOR  equations 
should  not  be  used  for,  and  what  equations  should  be  used  for 
these  other  interactions.  We  believe  that  the  key  word  in  the 
above  sentence  from  Reference  [10]  is  ’’Initially.”  We  recom- 
mend to  a potential  user  of  ATACM  that  he  not  accept  the  equa- 
tions initially  used  in  ATACM  (we  believe  they  have  several 
important  limitations),  and  that  he  select,  from  the  various 
types  of  attrition  equations  that  are  available,  those  which 
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he  feels  are  most  appropriate  for  the  interactions  modeled  in 
ATACM.  Various  suitable  attrition  equations  can  be  found  in 
References  [1],  [2],  [3],  [12],  and  [IM].  Section  (2)  of 
Reference  [1]  gives  a reasonable  menu  of  homogeneous  attrition 
equations  suitable  for  modeling  air  combat.  The  other  refer- 
ences provide  details  (such  as  how  to  handle  heterogeneous 
forces)  on  these  equations. 

Finally,  we  note  that  (a)  the  method  for  considering 
sortie  rates  described  on  page  17  of  Reference  [^]  will  not 
work  if  the  sortie  rates  are  less  than  1.0  (and  it  is  only  an 
approximation  if  the  sortie  rates  are  greater  than  1.0)  (see 
Reference  [3]  for  details);  (b)  aircraft  in  ATACM  are  sheltered 
proportionally,  not  by  priority;  (c)  ABA  aircraft  in  ATACM  know 
which  shelters  are  empty  and  they  attack  only  full  shelters 
(however,  aircraft  in  the  open  are  apparently  not  attacked 
preferentially  over  aircraft  in  shelters);  and  (d)  ATACM  does 
not  play  Quick  Reaction  Alert  aircraft. 

D.  A SUGGESTION  FOR  IMPROVING  ATACM 

If  the  assessment  methodology  were  sufficiently  Improved, 
we  would  have  only  one  criticism  of  ATACM;  namely,  the  MaxMln, 
MinMax  methodology  of  ATACM  is  meaningless  if  M - m is  large. 

And  M - m must  necessarily  be  large  for  those  sets  of  input  data 
for  which  the  difference  between  the  true  MinMax  and  true  MaxMin 
values  (M-m)  is  large  (l.e.,  the  games  being  played  do  not  have 
saddlepoint s , and  are  not  close  to  having  saddlepoints) . For 
these  sets  of  data,  we  believe  that  the  MaxMln,  MinMax  approach 
of  ATACM  should  not  be  used.  Either  a more  aggregated  model 
such  as  OPTSA  (Reference  [3])  is  required,  or  the  grid  points 
should  be  used  to  develop  approximate  optimal  strategies  in  a 
manner  similar  to  that  described  in  Reference  [7]. 

Since  it  is  generally  not  known  in  advance  whether  or  not 
M - m is  small,  we  recommend  either  using  OPTSA  or  implementing 
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concepts  similar  to  those  described  in  Reference  [7]  over 
improving  the  MaxMin,  MlnMax  approach  in  ATACM. ^ However, 
if  for  a particular  set  of  input  data  M - m were  small,  then 
ATACM  would  be  a worthwhile  model  to  determine  optimal  strate- 
gies (for  these  data)  provided  that — 

(1)  M - m were  shown  to  be  small,  and 

(2)  the  computer  running  time  of  ATACM  were  not  excessive 

But,  we  believe  that  even  if  M = m,  M - m will  not  usually  be 
small  if  the  rounding  procedure  described  here  (and  on  page  32 
of  Reference  [9])  is  used. 


Using  the  notation  defined  in  Reference  [9],^  as  well  as 
new  notation  defined  below,  we  suggest  the  following  alterna- 
tive to  ATACM' s rounding  procedure.  The  rounding  procedure 
is  used  in  ATACM  to  compute  a bound  on  TP^^^(Z^j (X^) ) . 
TP^^^(Z^j (X^) ) could  not  be  computed  directly  because  Z^^(X^) 
is  not  in  general  a grid  point.  So  Z^j(X^)  is  rounded  to 
nearest  "lower  right"  grid  point,  denoted  by  Z..(X. ),  and 
TP^_j_^(Z^j  (X^) ) is  used  in  place  of  TP^_j_^(Z^j  (X^ ) ) . We  recom- 
mend "rounding  the  strategies"  but  not  "rounding  the  resources 
That  is,  let 

S^(X  ) = Red's  one-stage  MlnMax  strategy  corresponding 
^ ^ not  to  X , but  to  the  grid  point  closest  to  X 

(not  necessarily  the  "upper  left"  grid  point). 


Let  ATACM  make  a "first  pass"  as  described  on  pages  30  and  32 
of  Reference  [9]*  On  the  second  pass  compute  ^ 


^The  description  of  the  DYGAM  model  in  References  [6]  and  [7]  emphasizes 
using  the  grid  point  approximations  to  allow  many  stages  to  be  played. 
We  believe  that  using  approximations  to  allow  playing  multiple  aircraft 
types  and  multiple  missions  per  aircraft  (as  claimed  in  the  features  of 
ATACM)  and  playing  range-payload  tradeoffs  are  relatively  much  more 
inportant  for  air  combat  models. 

^Except  that  we  replace  the  erroneous  notation  TP(X^)  and  S(X^)  with 
TP^(X^)  and  S^(X^),  respectively. 
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by  having  Blue  play  his  first  pass  strategy  (§)  and  by  Red 
playing  ^ ^ period  t+1,  and  playing  S^+2^^t+2^ 
on  period  t+2,  where 


and  playing  period  t+3,  where 


’'t+3  ' ^§t+2St+2'’'t+2 


r /v-r  ’ 


and  so  on  to  the  end  of  the  war.  That  is,  for  each  grid  point 
in  each  period,  the  war  would  have  to  be  re-fought  forward  to 
its  end  in  order  to  calculate  the  payoff  (in  the  same  manner  as 
the  war  is  fought  forward  to  determine  the  MaxMin  versus  MinMax 
payoffs  as  described  on  page  3^  of  Reference  [9])* 

This  suggestion  requires  three  comments.  First,  while  it 
will  not  necessarily  produce  true  bounds  on  the  MaxMin  and  Min- 
Max  values  of  the  game  described  in  Reference  [9],  it  will 
produce  true  bounds  on  the  game  where  the  commanders  are  re- 
quired to  make  the  same  allocations  when  between  grid  points 
as  they  do  on  the  nearest  grid  point.  For  example,  if  the  grid 
points  are  0,  ^00,  800,  and  1200,  and  a commander  has  5^8  air- 
craft, he  thinks:  "5^8  is  about  iJOO,  so  I'll  do  what  is  optimal 
for  ^00  aircraft"  (but  the  assessments  are  calculated  using  5^8 
aircraft).  Thus,  true  bounds  would  be  obtained  for  a slightly 
revised  game. 

Second,  this  suggestion  will  not  necessarily  make  M - in 
small,  even  if  M = m,  because  of  the  first  pass  approximation. 

Third,  this  suggestion  may  violate  condition  (2)  above, 
i.e.,  it  may  make  the  computer  running  time  of  ATACM  excessive. 
But  it  is  the  only  way  that  we  currently  see  to  enable  ATACM 
to  make  M - iii  small,  given  that  M - m is  small. 
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III.  COMPUTATION  OF  MAXMIN  AND  MINMAX 
STRATEGIES  IN  MULTI-STAGE  GAMES 


Eleanor  L.  Schwartz 

This  chapter  considers  multi-stage  games,  MaxMin  and  MinMax 
strategies,  and  computation  time.  The  OPTSA  multi-stage  game 
models  are  documented  in  detail  in  References  [3]  and  [il];  the 
reader  is  assumed  to  have  some  familiarity  with  them.  The  frame- 
work is  a game  with  a specified  number  of  decision  periods;  at 
the  beginning  of  each  decision  period  Blue  and  Red  each  (simul- 
taneously) make  a decision  from  an  input  list  of  strategies.  A 
stream  of  such  decisions  made  by  both  players  over  the  course  of 
the  game  leads  to  a final  payoff.  Methods  for  finding  decisions 
that  in  some  sense  optimize  the  payoff  are  sought;  traditionally, 
an  equilibrium  point  in  randomized  strategies  has  been  sought. 

The  idea  of  finding  MaxMin  and  MinMax  values  and  strategies  as 
enforceable  bounds  on  the  final  payoff  has  been  proposed  and 
implemented  in  ATACM  (Reference  [9])- 

At  all  times  the  amount  of  computation  needed  to  process  a 
game  must  be  considered.  Models  such  as  DYGAM  (References  [6] 
and  [7])  and  ATACM  find  approximations  (that  might  or  might  not 
be  close)  to  the  randomized  strategy  equilibrium  point  and  Max- 
Min and  MinMax  values,  respectively,  in  order  to  process  games 
of  more  stages  than  OPTSA  can  process.  This  approximation  is 
made  possible  by  computing  in  the  state  space  of  aircraft 
resources,  rather  than  the  space  of  possible  decisions. 

Section  A derives  an  efficient  way  to  find  MaxMin  and  Min- 
Max strategies  of  matrix  games,  bringing  together  the  game- 
solving method  of  the  Revised  OPTSA  Model  (Reference  [3])  with 
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MaxMln  and  MinMax  ideas.  Section  B discusses  MaxMin  and  MinMax 
strategies  in  multi-stage  games.  Section  C derives  computation 
time  formulas  for  finding  MaxMin  and  MinMax  strategies  for  the 
game  encountered  in  the  OPTSA  models. 

A.  COMPUTATION  OF  MAXMIN  AND  MINMAX  STRATEGIES  AND  VALUES 

OF  ONE-STAGE  GAMES 

The  Revised  OPTSA  Model  uses  a game-solving  algorithm  that 
is  able  to  find  optimal  randomized  strategies  for  one-stage 
matrix  games  without  computing  all  the  payoff  entries.  As  a 
result  the  total  running  time  is  greatly  reduced  from  previous 
versions.  This  method  can  be  adapted  to  find  MaxMin  and  MinMax 
strategies  of  a matrix  game  without  computing  all  the  payoff 
entries . 

The  problem  of  finding  the  optimal  randomized  Blue  strategy 
can  be  formulated  as  the  linear  program 

maximize  o 


m 

” - • 

J = 1 

to  n 

m 

l 1 

i=l  ^ 

1 0 

i = 1 

to  m 

where  a^j  is  the  payoff  to  Blue  when  Blue  chooses  pure  strategy  i 
(1  to  m)  and  Red  chooses  pure  strategy  J (1  to  n),  and  x^  is  the 
probability  Blue  plays  pure  strategy  i.  In  the  Revised  OPTSA 
Model  this  LP  is  solved  by  considering  a subset  of  the  first  n 
constraints,  solving  the  relaxed  LP,  and  checking  the  solution  to 
see  if  any  constraints  not  considered  are  violated.  If  none  are, 
the  current  solution  is  optimal  for  the  whole  game.  Otherwise 
the  most  violated  constraint  is  added  to  the  constraint  set  and 
the  procedure  repeated. 
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The  MaxMin  problem  can  be  formulated  as 


maximize  o 

m 

s.t.  o<  y a..x,>  j=lton 

- 1^1  10  1 


m (2) 

I ^1  = 1 

i=l  ^ 

x^  = 0 or  1 , i = 1 to  m. 


The  additional  constraints  that  the  x^^  be  integer  have  been  added 
to  the  original  LP;  therefore  the  optimal  objective  function  value 
will  be  lower.  Exactly  one  x^  will  equal  1;  the  corresponding  i 
is  Blue's  MaxMin  strategy. 


This  integer  program  can  be  solved  by  an  algorithm  similar 
to  the  relaxation  technique  above.  The  difference  is  that  in 
problem  (1)  each  relaxed  LP  was  solved  by  the  dual  simplex 
method;  here  a simple  comparison  test  can  be  used.  The  algo- 
rithm is  as  follows.  Let  Jq  c n}  be  the  set  of  Red  pure 

strategies  being  considered  at  the  current  iteration,  1q  the 
current  Blue  optimal  strategy,  and  Oq  the  current  objective 
function  value  to  the  relaxed  problem  (2).  This  solution  satis- 
fies the  constraints  in  J^,  l.e.,  for  j e J^, 


m 

1 I 

i=l 


^ij^l  = 


'’I 

^0'^  0 


= a 


IqJ 


Since 


for  the  feasibility  test  we  only  need  to  compute  the  elements 
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where 


Jq  — Jq  j 

and  compare  each  element  with  Oq.  If 

m 

for  each  j e Jq,  i.e.,  the  current  solution  is  also  feasible 
for  the  constraints  not  explicitly  considered,  problem  (2)  has 
been  solved,  as  addition  of  new  constraints  cannot  increase 
the  objective  function  value. 

Suppose  there  is  some  j^  ^ that  ^ ^i  j ‘ 

us  assume  also  that  ^ ^ 

i • J = 1 n . 

i.e.,  j^  is  the  constraint  most  violated  by  the  current  solution. 

We  can  compute  the  column  j^  and  augment  the  constraint  set  by 

letting  = Jq  U {j^^}.  We  now  solve  the  relaxed  problem  (2) 

with  constraint  set  J, . Let  us  assume  that  a minimum  value, 

0 ^ 

denoted  by  m . , of  each  row  over  the  columns  of  J^  has  been  kept. 

1 ^0 
Each  element  of  column  jj^  is  tested  in  turn.  If  a^  j ^ ^i  » 

1 10  ^ 
then  m = a.  ; if  not,  m.  = m.  . That  is,  given  the  minimum  of 

1 X X 

each  row  over  the  set  of  columns  Jq,  we  find  the  minimum  of  each 
row  over  the  set  of  columns  J^.  The  maximum  element  of  this  set 
of  row  minima  becomes  the  new  optimal  objective  function  value 
o^,  and  the  row  i^  in  which  it  appears  becomes  the  new  MaxMin 
strategy.  The  feasibility  test  is  then  performed  again;  stopping 
if  and  1^  are  optimal  for  the  whole  problem;  otherwise  the 
procedure  is  repeated. 

Problem  (2)  is  easier  to  solve  than  problem  (1)  in  that  no 
simplex  pivots  are  done,  merely  a comparison  test.  However, 
recall  that  solving  problem  (1)  also  found  the  Red  optimal 


randomized  strategy  from  the  dual  variables  at  optimality.  The 
Red  MinMax  strategy  and  value  cannot  be  directly  found  by  solv- 
ing problem  (2).  A similar  but  separate  problem  must  be  solved, 
By  suitable  storage  of  payoff  entries,  however,  duplications  in 
payoff  computation  can  be  avoided. 

An  example  of  the  new  method  of  solving  problem  (2)  will 
clarify  things.  This  game,  where  m = ^ and  n = 6,  was  used  as 
an  example  in  the  Revised  OPTSA  Model  documentation.  There  is 
no  saddlepolnt.  Looking  at  the  whole  matrix,  the  Blue  MaxMin 
strategy  is  B^l,  with  value  il. 


Step  la. 


R1 

R2 

R3 

Ri| 

R5 

R6 

B1 

5 

8 

1 

2 

H 

5 

B2 

5 

11 

3 

1 

1 

7 

B3 

3 

3 

4 

6 

1 

7 

B^4 

6 

10 

5 

8 

6 

Arbitrarily  let  R1 

be  1 

the  first  Red 

[ pure  strategy  to 

be  tried. 

SO 

Jo  = 

{Rl} 

Compute  column  Rl 

(elements 

^11  ^^1  ^ ’ 

resulting 

in  the 

matrix  below. 

where  a 

circle  around 

a payoff 

entry 

indicates  that 

it  has 

been  computed 

• 

R1 

R2 

R3 

R^ 

R5 

R6 

B1 

® 

8 

1 

2 

5 

B2 

© 

11 

3 

1 

1 

7 

B3 

© 

3 

6 

1 

7 

BiJ 

® 

10 

5 

8 

6 

The  minimum  row  values 

m^  are 

Just 

the  a.^. 

that  is. 

5,  5,  3, 

and 

6.  The  maximum 

of  these  is  6, 

occurring 

in  row  B^l 

, so 

^0  = 

ai|i 

= 6 and  i^  = 

25 


step  lb.  Perform  the  feasibility  test.  We  know  Oq  ± ^l|i» 


^ ^i|  i ’ 

J = 2 

to 

6?  ( 

Compute  row 

Bi| , yielding  the 

matrix  below. 

R1 

R2 

R3 

Ri)  R5 

R6 

B1 

© 

8 

1 

2 

5 

B2 

© 

11 

3 

1 1 

7 

B3 

© 

3 

6 1 

7 

Bi| 

© 

10 

5 

8 

6 

Note  that 

entry 

= 4 

VO 

II 

o 

t) 

V 

Therefore  Red  pure 

strategy  should  also  be  considered. 


Step  2a.  Let  the  constraint  set  = Jq  U (R^)  = {R1,R^}. 

Compute  column  R^  = 1 = 1 to  yielding: 


R1 

R2 

R3 

rh 

R5 

R6 

B1 

© 

8 

1 

® 

5 

B2 

© 

11 

3 

© 

1 
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B3 

© 

3 

i4 

© 

1 

7 

Bi< 

© 

© 

© 

® 

© 

We  now  compute  new  row  minima  mK  by  taking  the 
0 ^ 

minimum  of  m^  and  i =•  1 to  as  in  the 

table  below. 


Minimum  of 
row  i over  columns 


^ Jq  - 

15  2 

2 5 1 

3 3 6 

H 6 H 


Minimum  of 
row  i over  columns 


2 

1 

3 

ij 
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The  maximum  of  the  m^  is  in  row  . Thus  = ^ , which  is 
less  than  = 6,  and  i^,  the  MaxMin  strategy,  is  which 
happens  to  be  the  same  as  i^. 

Step  2b.  Perform  the  feasibility  test.  Is 

know  £ ^iji’  Since  row  H has 

already  been  computed  we  only  need  to  look  across 
and  see  that,  indeed,  is  the  smallest  element. 
Therefore  the  procedure  stops;  the  MaxMin  strategy 
is  Blue  pure  strategy  and  the  MaxMin  value  is  . 

Note  that  only  12  of  the  2^  payoff  entries  need  to 
be  computed. 

This  method  is  applicable  to  any  matrix  game.  It  could 
be  a huge  nonadaptive  game  where  each  row  and  column  repre- 
sents a stream  of  decisions  over  several  stages,  or  it  could 
be  the  small  matrix  occurring  at  one  stage  of  a behavioral 
game.  The  OPTSA  I games  could  be  solved  by  the  new  method, 
with  much  less  computation. 

An  Important  factor  to  consider  is  the  relationship  between 
the  number  of  rows  and  columns  that  need  to  be  computed  to  solve 
either  problem  (1)  or  problem  (2)  and  the  size  of  the  payoff 
matrix.  In  one  recent  analysis  that  used  OPTSA  and  solved  games 
for  many  variations  of  a realistic  set  of  data,  only  one  or  two 
rows  and  one  or  two  columns  were  required  to  solve  the  majority 
of  the  games.  This  is  probably  because  of  dominations  among 
the  pure  strategies  which  is  due  to  the  data  and  assessments 
used  in  OPTSA.  Payoff  matrices  generated  from  realistic  data 
may  have  enough  domination  among  rows  and  columns  so  that  the 
number  of  rows  and  columns  that  need  to  be  computed  grows  very 
slowly  with  the  size  of  the  matrix.  Even  payoff  matrices  with 
independent,  randomly  generated  elements  frequently  might  not 
require  computation  of  all  rows  and  columns  for  solution. 
Exploration  of  this  relationship  could  be  considered  in  further 
research  on  multi-stage  games. 
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B.  COMPUTATION  OF  NONADAPTIVE  AND  ADAPTIVE  MAXMIN  AND 
MINMAX  STRATEGIES  AND  VALUES  OF  MULTI-STAGE  GAMES 


Suppose  there  Is  a two-person  zero-sum  game  of  several 
stages,  containing  rules  about  when  Blue  and  Red  can  make  deci- 
sions and  what  information  each  side  has  at  any  decision  point. 
Given  these  rules,  suppose  Blue  wishes  to  formulate  (at  the 
beginning  of  the  game)  a decision  policy  to  guarantee  that  the 
payoff  is  at  least  some  value  a — no  matter  what  Red  does. 
Furthermore,  suppose  Blue  wishes  to  find  the  maximum  o for 
which  such  a decision  policy  exists.  Similarly,  suppose  Red 
wishes  to  find  a decision  policy  to  guarantee  that  the  payoff 
is  no  more  than  x,  no  matter  what  Blue  does,  and  to  find  the 
minimum  x.  The  maximal  o is  less  than  or  equal  to  the  minimal 
X,  and  they  may  be  equal.  For  a one-stage  matrix  game  (or  any 
game  in  normal  form)  the  standard  formulas 


o 


max 

1=1  to  m 


min 

j=l  to  n 


a 


ij 


and 


X = min  max  a 

j=l  to  n 1=1  to  m 


yield  the  maximal  a and  minimal  x,  and  their  arguments  i and  J, 
respectively,  provide  the  Blue  and  Red  decision  policies.  That 
is,  i is  such  that 


min  af.  = a 
J=1  to  n 

and  J is  such  that 


max  a . c = X . 

1=1  to  m ^ 

A matrix  game  such  as  this  can  be  solved  relatively  quickly  by 
the  method  in  Section  A above. 
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The  nonadaptive  multi-stage  game  of  OPTSA  I is  easily 
transformed  into  a large  one-stage  matrix  game  where  each  stra- 
tegy consists  of  a sequence  of  allocation  choices,  one  for  each 
period,  that  is  to  be  played  regardless  of  what  the  other  side 
does.  If  there  are  allocation  choices  available  to  Blue  in 
decision  period  k,  k = 1 to  s,  then  the  total  number  of  Blue  pure 

s 

strategies  for  the  large  matrix  game  is  B = II  b.  . The  matrix 

k=l  ^ 

game  has  the  following  quantities  associated  with  it. 

• A Blue  MaxMin  strategy  Ij^  and  associated  MaxMln  value  Oj^, 

• A Red  MinMax  strategy  and  associated  MinMax  value  Tj^, 

• A payoff  a.  ^ when  each  side  plays  its  nonadaptive 

strategy. 

(Note  that  o»r  1 a,  . < t„.) 

N ^ 

/S*p  /N  /\  /V 

• A game  value  g = x Ay,  where  x and  y are  the  optimal 
Blue  and  Red  randomized  strategies  for  the  nonadaptive 
game. 

(Note  that  £ g £ Tj^.  ) 

/s  ✓s 

The  value  g is  found  in  OPTSA  I (with  x and  y) ; one  cannot  say 

where  g lies  with  respect  to  a.  . (unless  = t„) . 

In^N  w in 

The  multi-stage  game  solved  in  OPTSA  II  allows  both  sides 
at  each  stage  to  choose  their  strategy  dependent  on  what  both 
have  done  during  preceding  stages.  Blue  (and  Red  too)  is 
allowed  to  formulate  a policy  based  on  all  the  freedom  of 
decision  he  has.  How  do  we  find  MaxMln  and  MinMax  strategies 
and  values  for  this  game?  First,  note  that  what  will  actually 
occur  in  the  game  is  a sequence  of  allocations  by  both  sides 
that  will  lead  to  a final  payoff.  Therefore  any  outcome  to  be 
taken  into  consideration  in  solving  an  adaptive  or  behavioral 
game  will  also  appear  in  the  corresponding  nonadaptive  game. 
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Therefore,  the  maximal  a,  call  it  o^,  is  at  least  as  there 
is  a strategy  for  Blue  (i.e.,  ij^)  by  which  he  is  assured  Oj^ 
with  certainty.  Examples  have  been  constructed  where  is 
strictly  greater  than  Similarly,  for  Red  there  is  a £ 

Tjj.  Corresponding  to  and  are  behavioral  strategies  that 
take  into  account  the  decision  rules  of  the  game.  (These  stra- 
tegies do  not  Involve  randomization.)  The  values  provide  bounds 
on  the  OPTSA  II  game  value  that  might  be  tighter  than  the  non- 
adaptive  MaxMin  and  MlnMax  values.  However,  it  is  not  possible 
in  general  to  tell  where  the  OPTSA  II  game  value,  the  OPTSA  I 
game  value,  the  outcome  from  each  side  playing  its  conservative 
nonadaptive  strategy  (MaxMin  or  MlnMax,  as  appropriate),  and 
the  outcome  from  each  side  playing  its  conservative  adaptive 
strategy  lie  with  respect  to  each  other. 

Putting  the  game  of  OPTSA  II  into  normal  form  would  Involve 
listing  all  the  policies  for  Blue  and  Red.  This  would  result  in 
a huge  unwieldy  matrix.  However,  the  method  used  to  find  the 
optimal  (randomized)  behavioral  strategies  can  be  combined  with 
the  method  in  Section  A above  to  find  (in  two  passes)  the  values 
and  and  the  associated  MaxMin  and  MlnMax  behavioral  stra- 
tegies. Let  us  consider  a game  of  three  stages  for  an  example, 
and  let  there  be  bj^  "pure"  choices  for  Blue  and  rj^  for  Red  at 
stage  k,  k = 1 to  3-  At  the  end  of  stage  2 there  are 
places  where  Blue  could  be,  one  for  each  possible  history  of 
choices.  At  each  of  these  places  Blue  is  faced  with  a one-stage 
matrix  game  for  which  the  MaxMin  strategy  and  value  can  be  found 
by  the  method  in  Section  A.  This  results  in  t>^b2r^r2  values 
each  guaranteed,  conditioned  on  the  place  at  the  end  of  stage  2. 
Organize  these  into  matrix  gcunes,  each  x For  each 

of  these  matrix  games,  the  MaxMin  strategy  will  guarantee  Blue 
the  maximum  value  possible  regardless  of  where  Red  moves,  condi- 
tioned on  the  outcome  at  the  end  of  stage  one.  This  yields 
MaxMin  values,  which  are  organized  into  one  final  game,  the  Max- 
Min strategy  and  value  of  which  yield  Blue's  first  period  choice 
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and  the  value  a^.  To  find  the  Red  MinMax  strategies,  the  same 

stage  2 can  be  used,  but  the 

b^r^  matrices  of  MinMax  values  will  not  be  the  same  as  the 
matrices  of  MaxMin  values;  therefore  some  kind  of  two-pass 
procedure  is  needed. 

The  embedding  procedure  of  the  Revised  OPTSA  Model  (Refer- 
ence [2],  Chapter  II,  Section  E)  can  be  used  to  find  the  adaptive 
MaxMin  and  MinMax  strategies  as  efficiently  as  possible.  With 
judicious  programming  the  two  passes  of  the  procedure  can  be 
meshed  to  avoid  unnecessary  duplication  in  computation. 

The  motivation  of  both  the  DYGAM  and  ATACM  models  is  to 
find  quantities  connected  with  the  game  of  OPTSA  II  where  the 
numbers  bj^  and  r^^  are  so  large  that  OPTSA  II,  or  the  adaptive 
MaxMin  and  MinMax  finding  procedure,  would  take  too  much  com- 
puter time.  They  both  make  the  approximation  of  condensing  the 
"^1^2^1^2  stage  2"  into  somewhat  fewer 

places.  DYGAM  tries  to  find  the  optimal  randomized  behavioral 
strategies;  ATACM  tries  to  find  the  adaptive  MaxMin  and  MinMax 
strategies.  Because  of  this  condensation  of  the  state  space, 
games  with  many  more  stages  than  3 can  be  processed  in  a reason- 
able amount  of  computer  time. 

C.  COMPUTATION  TIME  REQUIREMENTS 

This  section  derives  estimates  of  computation  time  formulas 
for  several  different  games  solved  by  OPTSA  methods.  The  ap- 
proach is  similar  to  that  used  in  Reference  [3].  The  formulas 
are  in  terms  of  number  of  days  of  combat  simulated,  since  the 
running  time  of  all  the  OPTSA  models  is  essentially  proportional 
to  that  number.  Given  the  time  it  takes  to  simulate  one  day  of 
combat  in  an  assessment  routine,  an  estimate  of  the  time  OPTSA 
would  take  to  solve  the  game  can  be  obtained.  This  can  then 
reasonably  be  compared  with  the  time  of  other  models. 
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Let  there  be  s stages  and  let  and  be  the  number  of 
pure  strategies  available  to  Blue  and  Red,  respectively,  in 
stage  k.  Let  stage  k comprise  dj^  days  of  combat;  thus  a war 
s 

has  D = d,  days.  The  first  days  of  the  stages  are  decision 
k=l  ^ 

days,  which  must  be  identical  for  Blue  and  Red.  Let  w and  c be 
average  estimates  of  the  number  of  rows  and  columns  that  need  to 
be  computed  to  solve  a matrix  game.  The  actual  number  needed 
might  vary  somewhat  with  the  input  data  and  advance  start . As- 
sume w and  c are  independent  of  the  size  of  the  matrix  (as  has 
been  discussed);  also  assume  they  are  independent  of  stage.  The 
two-pass  MaxMin  and  MinMax  procedure  might  yield  values  of  the  w 
and  c different  from  those  found  in  the  procedure  for  finding 
the  optimal  randomized  strategy. 

1 . Computing  Randomized  Strategies  for  Nonadaptive  Games 

s 

There  is  one  large  matrix;  the  order  is  II  b,  rows  by 

n r.  columns.  Call  these  numbers  B and  R,  respectively.  The 
k=l  ^ 

number  of  payoff  entries  that  need  to  be  generated  is  wR  + cB  - 
wc.  The  last  term  will  be  insignificant  if  B and  R are  large. 

To  find  the  number  of  daily  campaigns  needed,  first  note  that 
the  procedure  described  in  Reference  [3]  can  be  applied.  This 
involves  not  resimulating  the  whole  war  but  only  the  latter 
periods  where  the  pure  strategy  changes.  That  is,  to  compute 
a column  of  R elements  one  would  think  that  RD  dally  campaigns 
would  be  required.  However,  this  can  be  reduced  considerably 
s k 

to  51  d n r..  For  example,  if  s = 3 and  r.  = n v A,  the 

k=l  ^ £=1  ^ ^ 

3 2 3 D 

formula  reduces  from  n D to  nd,  + n d^  + n d^;  if  d,  = — for 

1 2 3 is 

s-1 

1 = 1 to  s,  the  reduction  is  on  the  order  of  . The  corres- 

’ s 

ponding  formula  for  the  number  of  daily  campaigns  needed  to 
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compute  a row  of  B elements  reduces  from  BD  to 


With  w rows  and  c columns  needed  to  be  computed,  the  total 
formula  becomes: 


Number  of  daily  campaigns  that  must  be  computed  « 


campaigns  per  row  cairpaigns  per  column 


where 

w = number  of  rows, 
c = number  of  columns. 

For  some  (though  not  most)  assessment  routines,  the  method  of 
recomputing  only  the  days  where  the  strategy  changes  may  not 
be  valid.  Thus  wBD  + cRD  remains  a general  estimate,  though 
an  extremely  conservative  one. 


2 . Computing  Randomized  Strategies  for  Adaptive  Games 


This  formula  is  derived  in  Reference  [3];  we  restate  it 
here.  For  stage  J,,  let  a^^  = wr^^  + cb^^  - wc.  This  is  an  esti- 
mate of  the  number  of  payoff  entries  needed  to  solve  a game  at 
stage  Z.  The  number  of  daily  campaigns  needed,  using  the  ”no 
unnecessary  days  recomputed"  feature,  becomes  ^ a^a2d2+...+ 


(^) 


The  estimates  for  the  MaxMin  and  MinMax  games  are  exactly 
the  same  for  randomized  strategy  ones,  except  w and  c might  be 
bigger  if  two  passes  are  made.  This  is  because  exactly  the 
same  kind  of  relaxation  technique  is  used;  the  difference  is 
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between  simplex  pivots  and  comparison  tests,  which  is  very 
small  compared  to  payoff  computation  time.  Thus  formula  (3) 
serves  also  for  nonadaptlve  MaxMin  and  MinMax  games,  formula 
(4)  for  adaptive  ones. 

These  formulas  can  be  simplified.  Let  b^  = r^^  = n for  all 
i (S.=l  to  s),  and  call  - wr^  + cbj^  - wc  = a for  all  H.  Then 
B = R = n®.  Formula  (3),  the  nonadaptlve  game,  then  reduces  to 
s , 

(w+c)  'l  d,^n^.  If  d is  large  compared  to  the  other  d,  , as  is 
k=l  ^ ® ^ 

often  the  case,  the  sum  is  dominated  by  its  last  term,  d n®. 

s 

r k 

Formula  (4)  becomes  ) ^ d,  , which  again  is  dominated  by  its 

k=l  ^ 

last  term  a^d  . The  ratio 
s 

a®d  s 

adaptive  # campaigns  s _ 1 fa-j 

nonadaptlve  § campaigns  ~ (w+c)n^d  (w+c ) '■n-' 

s 

Given  w and  c,  — is  almost  Independent  of  n.  If  w = c = 2, 
then  a = wn  + cn  - wc  = 4n  - 4 so  a/n  w 4.  As  s gets  larger 
the  nonadaptlve  game  quickly  becomes  much  more  favorable  in 
terms  of  computation. 


IV.  PURE  STRATEGIES  IN  ADAPTIVE  AND  NONADAPTIVE  GAMES 
James  E.  Falk  and  Jeffrey  H.  Grotte 

In  general,  research  In  multi-stage  games  has  dealt  with 
mixed-strategy  phenomena.  The  foregoing  chapters  on  the  other 
hand,  raise  questions  concerning  the  features  of  pure  strate- 
gies in  multi-stage  games.  The  beginnings  of  two  parallel 
approaches  to  this  problem  are  given  in  Appendices  A and  B. 
While  the  content  of  these  appendices  overlap  to  some  extent, 
their  philosophy  and  notations  differ  and  consequently  it  is 
of  interest  to  present  them  both. 

The  key  theorem,  common  to  both  appendices,  proves  that 
if  a multi-stage  game  has  a nonadaptive  pure  strategy  saddle- 
point,  then  no  adaptive  strategy  saddlepolnt  will  yield  a 
better  MaxMin  value  to  the  maximizing  player  or  a better  Min- 
Max  value  to  the  minimizing  player.  Appendix  A also  gives  a 
condition  for  an  adaptive  strategy  saddlepolnt  to  imply  the 
existence  of  a nonadaptive  strategy  saddlepolnt  and  shows  how 
to  relate  the  adaptive  and  nonadaptive  strategies  when  adap- 
tive and  nonadaptive  saddlepoints  exist.  Some  examples  are 
also  presented.  Appendix  B,  in  addition  to  the  key  theorem, 
discusses  the  effectiveness  of  nonadaptive  strategies  against 
pre-announced  adaptive  strategies. 
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APPENDIX  A 


ON  ADAPTIVE  AND  NONADAPTIVE  PURE  STRATEGIES 
IN  ZERO-SUM  SEQUENTIAL  GAMES 

by 


Jeffrey  H.  Grotte 


APPENDIX  A 


ON  ADAPTIVE  AND  NONADAPTIVE  PURE  STRATEGIES 
IN  ZERO-SUM  SEQUENTIAL  GAMES 

Jeffrey  H.  Grotte 

Let  G be  a two-person  zero-sum  sequential  game;  i.e.,  G 
comprises  M stages.  At  each  stage  each  player  executes  an 
action  that  he  chooses  from  some  set  of  actions  available  to 
him  at  that  stage.  At  the  end  of  the  game,  each  player  will 
receive  some  payoff  based  on  both  players*  choices  of  actions. 
We  will  assume  that  each  player's  set  of  actions  at  any  stage 
is  finite  and  known  by  both  players  before  the  game  begins. 
Further,  a player's  set  of  actions  at  any  stage  is  independent 
of  either  player's  previous  actions.  For  the  purposes  of  this 
paper,  each  player  may  play  in  one  of  two  modes; 

Nonadaptive  pure  strategy . The  player  must  pick  his 
actions  for  each  stage  before  the  game  begins,  and  then  play 
his  sequence  of  actions  regardless  of  how  the  game  progresses. 

Adaptive  pure  strategy.  For  each  stage,  the  player  waits 
to  see  what  actions  both  sides  have  taken  up  to  that  stage 
before  choosing  his  action  for  that  stage.  For  this  case,  we 
will  require  that  the  player  have  perfect  and  complete  knowl- 
edge of  the  past.^ 

We  will  look  at  two  extremes  and  compare  the  results.  The 
cases  will  be: 


^Others  have  examined  adaptive  pure  strategies  in  other  contexts.  See  for 
exanple,  the  majorant  and  minorant  games  of  von  Neumann  and  the  meta- 
games of  Howard  [1].  The  reader  is  also  referred  to  Kair  [2]  and  Kuhn 
[3].  (These  references  are  listed  at  the  end  of  this  appendix.) 
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(1)  Both  players  play  nonadaptive  strategies. 

(2)  Both  players  play  adaptive  strategies. 


A remarkable  fact  is  that,  if  we  disallow  randomization, 
both  these  cases  can  be  modeled  by  normal  form  games. 


Let  i=l,2;  J=1,...,M  be  the  players'  sets  of  actions 

<3  i 

for  all  stages,  so  that  S.  is  player  i's  action  set  at  stage  j. 
1 J i 

Denote  by  s.  some  specific  element  of  S..  Thus  a play  of  the 
J J 

game  is 


fs^s^  s^-  s^s 


i oi 
s . e S . 
J J 


i=l,2;  j=l. 


,M 


The  payoff  of  the  game  (from  2 to  1)  we  denote  by 


0(si 


S ^ • • • s 


2*  • 


1. 

M’ 


2 2 
S1S2. 


■4) 


Nonadaptive  Model 

Let  = { (s^s^.  . .sjj)  I Sj  e sj-}  i=l,2  . 

Then  define  the  normal  form  game  GN  with  the  first  player's  pure 

strategies  being  the  elements  of  R^,  and  the  second  player's 

2 

pure  strategies  being  the  elements  of  R . We  define  the  out- 
come of  the  players'  choices  r e R^  i=l,2  in  the  following 
manner 


0{ 


1 1 
s ^ s 2 * • 


2 2 
s 1 s 2 * 


where 


s^s^ 

®1®2- 


s^s^. 
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Example 


A two  stage  game 


= {a,b} 


Let 


= {a,b} 

= {p,q} 

0^  = 0(aa,pp) 

(?2  = 0(aa,pq) 

0^  = C?(aa,qp) 

0^  = 0(aa,qq) 

0^  = 0(ab,pp) 

Og  = 0(ab,pq) 

Orj  = 0(ab,qp) 

Og  = 0(ab,qq) 


^2  ~ ^-PjqJ 

0^  = 0(ba,pp) 
O^Q  = 0(ba,pq) 
C>ii  = 0(ba,qp) 
(?12  = 0(ba,qq) 
0^2  = 0(bb,pp) 
0^2^  = 0(bb,pq) 

^15  " t^(bb,qp) 
O^g  = 0(bb,qq) 


then  the  normal  form  of  this  nonadaptive  case  is 


aa 

ab 

ba 

bb 


pp 

pq 

qp 

qq 

03 

^6 

O'l 

09 

^'10 

1 — 1 
r~i 

0 

0 12 

0 2i| 

"is 

O'le 

Figure  1 


Adaptive  Model 

Although  straightforward,  the  adaptive  model  requires 
somewhat  more  effort  to  formulate. 
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Define  and  r S^.  For  j=2,...,iyi,  let  Fj  be  the 

set  of  all  functions  f from 

X S2X...xS^_^  into  . 

Similarly  define  fj,  . 

Note  that  F^  is  a set  of  actions  while  F^,  j 5^  1 is  a set  of 
-L  i 

functions.  The  finiteness  of  the  guarantees  the  finiteness 

of  the  F^. 

J 

Now  define 


Now  we  can  model  the  adaptive  case  by  the  normal  form  game  GA 

where  player  I’s  pure  strategies  are  the  elements  of  and 
2 

where  P is  the  set  of  player  2’s  pure  strategies.  We  evaluate 

112  2 

the  outcome  of  a choice  p e P , p e P as  follows: 


p^  equals  some  sequence  f^f^.  . .fj^  and 

2 2 2 2 
p equals  some  sequence  f^f2...fjyj  . 

Now,  since  F^  = for  i = 1,2,  then  f^  = some  s^  e 1=1,2. 

We  can  now  find  inductively 


s 


1 

2 


" ^3(^15^)  s|  = f3(sjs^) 

sj  = fj(s^s2s^)  si  = f^(s];s^s^) 


and  so  on . 

Define  O^P  >P  ) = s^S2...  • 


A-^^ 


Example 


S^;  = {a,b}  s]  = {a,b> 


= {p,q> 


= ^p>q}  • 


Then 


= |a,b| 


.1  _ 


" ) ''a’  'b’  ^a’ 


b I 


where  defines  the  function 

fjCp)  = g 


f^Cq)  = h . 


Similarly  = {p,q},  F^  = {^/p>  ^/g>  Vp,  ^Z^}  where 


defines 


f^Ca)  = I f^Cb)  = m 


Using  our  procedure  for  computing  outcomes,  and  the  notation 
from  the  nonadaptlve  example.  It  Is  easy  to  see  that  the  normal 
form  game  we  have  thus  determined  Is  given  In  Figure  2. 

Note  the  obvious:  the  set  of  outcomes  of  the  game  GN  Is 

precisely  the  set  of  outcomes  of  the  game  GA,  not  counting 
multiplicities . 


LEMMA  1 

: a) 

max 
1 1 

min  r 1 2i 

2 2 ’P  J 

max  min  1 2 

- 1 „1  2 „2  '•  ’ 

p-^eP-^ 

p eP 

r eR  r eR 

b) 

min 

2 r,2 

p eP 

max  1 2> 

1 1 o(p  ,P  ) 

p eP 

min  max  ^ r 1 2 

-2  2 1 1 
r eR  r-^ER-*- 

PROOF: 

a)  Let  r^  be 

an  element  of 

R^  at  which  the 

max  min  i ? 1 

1 „2  2 „2  Dfr'^.r^l  Is  achieved.  We  know  r represents  the 

r eR  r eR  - 


1 

% 
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sequence 


P P/ 

P 

p ^ 
^ 'p 

q 

'P 

q 3- 

'p 

q S' 

9. 

a ^ 

0 

0 

0 

0 

0 

0 

0 , 

0, 

a 

1 

1 

2 

2 

3 

3 

H 

h 

a 

0 

0 

0 

0 

0 

OOr, 

Ors 

1 

1 

2 

2 

7 

'7 

8 

8 

^5 

S 

"6 

s 

s 

<>7 

^8 

^8 

So 

s 

^10 

•’ll 

^12 

^11 

0i2 

% 

% 

^10 

^15 

<>16 

‘>15 

^16 

^13 

S3 

^lil 

‘’ll 

^12 

Si 

(?12 

^^13 

^3 

(?ll, 

<’15 

Se 

S5 

^^16 

Figure  2. 


Select  that 


where 


f^(-)  = for  all  arguments,  . 

J J 

Observe  that  the  row  of  the  game  GA  must  have  precisely  the 
same  outcomes  as  the  r^  row  of  the  game  GN;  therefore 
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Therefore 


max 

r^eR 


1 


min 

2 2 
r eR 


max 

p^eP^ 


2i  - max  min  2\ 

2 2 J - 1 1 2 2 J 

r-^eR-^  r'^eR'^ 


Assertion  b)  Is  proven  in  an  entirely  analogous  manner. 

PROPOSITION : If  GN  has  a pure  strategy  saddlepoint,  so 

does  GA.  In  this  case  the  values  of  the  two  games  are  equal. 

PROOF : This  follows  trivially  from  inequalities  (a)  and 

(b)  of  the  Lemma. 


PROPOSITION: 


Let  (r^,r^) 


be  a pure  strategy  saddlepoint 


for  the  game  GN,  where 


1 


r 


11  1 
®1®2 ' ■ ■ ’ 


2 


r 


2 2 
s ^ s 2 • • 


s 


2 

M • 


Let 


1 


P 


^1  «1 
^1  2'  ' 


2 

P 


2 2 
f ^f 
^1^2* 


.f 


2 

M 


where 


f^(.)  = for  all  arguments,  j=2...M 

J J 

f?(*)  = s^  for  all  arguments,  j=2...M  . 

J J 

1 2 

Then  (p  ,p  ) is  a saddlepoint  for  the  game  GA. 

PROOF:  Clearly  o(p^,P^)  = o(r^,r^)  = value  of  GM  = 


value  of  GA. 


1 2 

We  have  yet  to  show  that  is  a saddlepoint 

1 ^12  ^ X 2 

Suppose  there  is  some  £ such  that  0(p  ,p  ) > d(p  ,p  ), 


NoWj  £ represents  the  sequence 

_1 

-1-2  • " ' 


Compute 


s^  = 

-1  -1 


and  so  on  until  s s 
Therefore 


1 - -fir  2'» 

^2  —2  ^^l'^ 

s have  been  computed. 


^^X2  X2  X 22  2 ^X2 

»p  ) “ ^ ( 5 . . . sjj ; S^S^.-.S^)  > 0(p  ,p  ) (by  assumption) 

= 0 (r  ,r  ) = ()  (s^S2  • • • Sj^;  s^S2*.-Sj^) 


So  that  there  is  some  r^  = 


X 2 

This  contradicts  the  assertion  that  (r  ,r  ) was  a saddle- 
point  for  GN. 

2 

In  a similar  way  one  proves  that  there  is  no  p such  that 

'^/•l  S') 

d(p  ,£  J < 0\,P  ,p  J . 


1 


What  we  have  shown  so  far  is  that  whenever  the  nonadaptive 
game  has  a pure  strategy  saddlepoint,  then  so  does  the  adaptive 
game.  Moreover,  when  this  occurs,  at  least  one  of  the  adaptive 
saddlepolnts  will  be  the  "image”  of  a nonadaptive  saddlepoint, 
hence  it  suffices  to  look  at  the  nonadaptive  game.  It  is  im- 
portant to  realize  that  knowing  that  the  adaptive  game  has  a 
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pure  strategy  saddlepoint  yields  no  information  about  the  non- 
adaptive  game  except  in  the  following  circumstance: 

X 2 

PROPOSITION:  If  GA  has  a pure  strategy  saddlepoint  (p  ,p  ) 

where 

1 _ 11  1 
P - i 2 • • • -L  jyj 

^2  _ „2-2  „2 

p - f^f2-  . .fjjj 


and  the  fj  have  the  property  that 


f^  * some  s^ 


2 2 
fj^  = some  s^ 


and 


fj(*)  = some  s^  for  all  arguments 


4-u  1 11 

then  if  r = s^S2--.Sj^ 
point  of  the  game  GN. 


1=1,2; 

1 


j=2,  . . ..,M 
2 


2 2 2 2 X 2 

r = s^s^.-.s^,  (r  ,r  ) is  a saddle- 


X 2 

PROOF ; Since  (p‘‘’,p^)  is  a pure  strategy  saddlepoint  of  GA 


0(p\p2)  = min  0(p^pO 
£^eP^ 


1 2^ 


But 


^X2  XX  X 22 

>P  ) 0(s2^S2»»«  Sj^  j ^ X ^ 2 


4)  ■ 


Therefore 


0 (sJ^S2. 


s^  s^s^  s^) 


= min 

2^c2 

-J 


0(s];s2. 


1.  2 2 
®M’  -1-2' 


•4) 
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or  equivalently 


min 

r^eR^ 


In  a similar  manner,  it  is  possible  to  show 


0 (r 


1 


max 

plcRl 


Hence 
its  column 


2 

(?(r'^,r‘^)  is  the  minimum  of  its  row,  the  maximum  of 
1 2 

so  that  (r'^,r‘^)  is  a pure  strategy  saddlepoint. 


EXAMPLES : The  following  examples  are  the  two-stage  game  with 

sj  = {a,b}  = {a,b} 

5^  = {p,q}  = {p>q}  • 

The  outcomes  can  be  found  in  the  table  for  the  nonadaptive 
case  of  each  example.  In  all  cases  e,y  > 0 

Example  1 shows  a game  in  which  the  nonadaptive  game  has 
a saddlepoint,  so  that  the  adaptive  game  also  has  one.  More- 
over, if  one  looked  only  at  the  adaptive  game,  one  would  notice 
that  one  of  the  saddlepoint s (in  the  upper  left  hand  corner) 
corresponds  to  a nonadaptive  strategy  pair  which  is  a saddle- 
point of  the  nonadaptive  game.  This  illustrates  the  final 
Proposition. 

Example  2 is  a case  where  the  nonadaptive  game  has  no 
saddlepoint,  although  the  adaptive  game  has  one. 

In  Example  3>  neither  game  has  a saddlepoint. 
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Example  1 


aa 


PP 


pq 


qp 


qq 


ab 


-e 


-E 


ba 


-e 


-e 


bb 


-e 


Nonadaptive  Case 


MaxMin  = 0 
MlnMax  = 0 
* saddlepolnt 


p V 

/p 

'>% 

0* 

0 

Y 

Y 

Y 

Y 

0* 

0 

« % 

0 

0 

Y 

Y 

0 

0 

-G 

-G 

-G 

Y 

Y 

Y 

Y 

0 

0 

->b 

-e 

-G 

Y 

Y 

0 

0 

-e 

-G 

-e 

-G 

-G 

-G 

0 

Y 

0 

Y 

•>  5^ 

-G 

-G 

-G 

-G 

-G 

Y 

-G 

Y 

* 

0 

Y 

0 

Y 

0 

Y 

0* 

Y 

0 

Y 

0 

Y 

-G 

Y 

-G 

Y 

MaxMin  = 0 MinMax  =0  * saddlepolnt 


Adaptive  Case 
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Example  2 


pp 

pq 

qp 

qq 

aa 

0 

Y 

-e 

0 

ab 

-e 

0 

Y 

Y 

ba 

-G 

Y 

-G 

bb 

Y 

-e 

Y 

0 

Nonadaptive  Case 


MaxMin  = -e 
MlnMax  = y 


p V 

/p 

q 

/p 

q y 

/q 

\cd 

cd 

0 

0 

Y 

Y 

-G 

-G 

0 

0 

0 

* 

0 

Y 

Y 

Y 

Y 

Y 

Y 

-G 

-G 

0 

0 

-G 

-G 

0 

0 

-G 

-G 

0 

0 

Y 

Y 

Y 

Y 

-G 

-G 

-G 

-G 

Y 

-G 

Y 

-Y 

-G 

-G 

-G 

-G 

Y 

0 

Y 

0 

Y 

-G 

Y 

-G 

Y 

-G 

Y 

-G 

Y 

-G 

Y 

-G 

Y 

0 

Y 

0 

MaxMin  = 0 MinMax  =0  * saddlepoint 

Adaptive  Case 
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Example  3 


pp 

pq 

qp 

qq 

aa 

0 

-e 

Y 

-e 

ab 

-e 

Y 

0 

^ MaxMin  = -e 

ba 

Y 

0 

Y 

-e  MinMax  = y 

bb 

-e 

Y 

-e 

0 

Nonadaptive  Case 


p 

^p 

p y 

/q 

q 5^ 

/q 

* 5$ 

0 

0 

-e 

-e 

Y 

Y 

-e 

-e 

0 

0 

-e 

-e 

0 

0 

Y 

Y 

-e 

-e 

Y 

Y 

Y 

Y 

-G 

-e 

-6 

-e 

Y 

Y 

0 

0 

Y 

Y 

Y 

0 

Y 

0 

Y 

-e 

Y 

-e 

Y 

0 

Y 

0 

-e 

0 

-e 

0 

-e 

Y 

-e 

Y 

Y 

-e 

Y 

-e 

-e 

Y 

-e 

Y 

-e 

0 

-e 

0 

MaxMin  = -e  MlnMax  = y 

Adaptive  Case 
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APPENDIX  B 

SADDLEPOINTS  OF  ADAPTIVE  GAMES 
James  E.  Falk 

In  this  appendix  we  examine  the  relationship  between  saddle- 
points  of  adaptive  games  and  saddlepolnts  for  their  associated 
nonadaptlve  games.  In  particular,  we  show  that  the  MaxMin  and 
MinMax  values  of  the  nonadaptlve  game  bound  the  same  quantities 
for  the  adaptive  game.  It  follows  that  the  adaptive  game  will 
have  a saddlepoint  if  the  nonadaptlve  game  has  one.  An  example 
is  given  which  shows  that  the  opposite  is  not  true,  i.e.,  the 
adaptive  game  may  have  a saddlepoint  while  the  nonadaptlve  game 
does  not. 


Notation: 


Let  n (number  of  periods)  be  given.  For  each  n,  let: 

A^(x^,y^,p^,q^)  = payoff  in  period  i when  in  state  (p^,q^) 

and  actions  x^  and  y^  are  chosen  by  the 
players , 

P^(x^,y^,p^,q^)  = state  p^^^  resulting  from  state  (p^,q^) 

when  actions  x^  and  y^  are  chosen, 

Q^(x^,y^,p^,q^)  = state  q^^^  resulting  from  state  (p^,q^) 

when  actions  x^  and  y^  are  chosen, 

X^,y^  = constraint  sets  on  decision  variables 
x^  and  y^  in  period  1, 

(p^,q^)  = given  initial  state. 


Then 


n 


A(x,y)  = I A^(x^,y^,p^,q^) 


i=l 
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where 


X = 


y 

)i+l 

i+1 


/.I  n. 

(x  ,x  ) 

/I  rix 

(y  , . • . ,y  ) 

1 1 1 i. 

P (x  ,y  ,p  ,q  ) 
i i i iv 

Q (x  ,y  ,p  ,q  ) 


is  the  cumulative  payoff  function  whose  value  is  uniquely  deter- 
mined when  the  players  choose  vectors  x and  y. 


In  order  to  define  the  game  G,  we  first  define  the  types  of 
strategies  available  to  the  players  I and  II.  An  adaptive  stra- 
tegy for  player  I is  a sequence  of  (vector-valued)  functions 

x^( ,x”( • ) 

where  x^(-)  : (p^,  q^)  -<■  X^.  Thus  an  adaptive  strategy  for 
player  I prescribes  an  action  or  decision  for  each  state  (p^,q^) 
that  can  be  realized  in  each  period  i.  A similar  definition 
holds  for  player  II. 

A nonadaptive  strategy  for  player  I is  a sequence  of 
vectors 

x^  x"^ 

where  x^  e X^.  Thus  a nonadaptive  strategy  for  player  I also 
prescribes  an  action  or  decision  for  each  period  i,  but  there 
is  no  state  dependence. 


Let 


S = set  of  all  nonadaptive  strategies  for  I, 

S = set  of  all  adaptive  strategies  for  I, 

and 

T = set  of  all  nonadaptive  strategies  for  II, 
T = set  of  all  adaptive  strategies  for  II. 
Note  S C S,  T C T. 

Player  I*s  adaptive  problem  is 

max  min  A(s,t)  = V 
seS  tel 
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and  his  nonadaptive  problem  is 

max  min  A(s,t)  = v . 
seS  teT 

Player  II 's  adaptive  problem  is 

min  max  A(s,t)  = V 
teT  seS 

and  his  nonadaptive  problem  is 

min  max  A(s,t)  = v . 
teT  seS 


We  have 


and 


V < V 


V < V . 


We  will  show  that 


V < V < V < V . 


Results : 


First,  we  prove  the  following  result: 

Theorem.  With  the  definitions  given  above,  if  s*^  = (x^(‘), 
. ..,x^(’))  is  any  adaptive  strategy  for  I,  then 

min  A(s*^,t)  = min  A(s^,t)  . 
teT  teT 


Proof . Since  T C T,  we  have 

min  A(s^,t)  £ min  A(s*^,t)  . 
teT  teT 

0*1 

Let  t^  = (y'^C  •),...  ,y"( ') ) solve^  the  left  of  these  two  problems. 


^Without  proper  assunptions  on  the  constituent  functions,  the  problem  may 
have  no  solution.  We  shall  assume  that  the  proper  assunptions  do  hold . 
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and  set 


and 


= x^(p^,q^)  = y^(p^,q^) 


p^  = P^(x^,y^,p^,q^) 

q^  = Q^(x^,y^,p^,q^) 


Continuing,  set 


and 


-2  2 , 2 2.  -2  2,  2 2. 
X = X (p  ,q  ) y = y (p  ,q  ) 


3 r>2,-2  -2  2 2x 

P = P (x  ,y  ,p  ,q  ) 

3 n2/-2  -222. 

q = Q (x  ,y  ,p  ,q  ) 


etc. , to  produce  two  sequences  of  decisions 


-1 


:rn 


,x 


and 


—1  — n 

y > • • • j y 


for  the  players.  (This  pair  of  sequences  is  often  termed  a 
play  of  the  game  G whose  payoff  is  A when  strategies  s and  t 
are  employed.) 


Note  that 


But 


t = (y^,. . . ,y")  e T . 


A(s*^,t*^)  = A(s^,t) 


since  the  same  play  of  the  game  is  produced  by  either  of  the 
pairs  (s^,t°)  or  (s*^,t).  Thus 


min  A(s*^,t)  = A(s*^,t*^)  = A(s*^,t) 
teT 

^ min  A(s^,t) 
teT 

^ min  A(s*^,t ) 
teT 

and  the  proof  is  complete. 

The  theorem  implies  that  player  II  can  restrict  himself  to 
his  nonadaptive  strategies  if  player  I announces  his  intent  to 
employ  a given  adaptive  strategy.  This  is  analogous  to  the 
situation  in  matrix  games,  wherein  a player  may  restrict  himself 
to  his  pure  strategies  if  his  opponent  announces  his  intent  to 
employ  a certain  mixed  strategy. 

Corol 1 ary . With  the  above  definitions 

max  min  A(s,t)  ^ max  min  A(s,t) 
seS  teT  seS  teT 

i. e.  , X.  1 Y. 

Proof.  Let  s denote  any  nonadaptive  strategy  for  I.  Then 

min  A(s,t)  = min  A(s,t) 
teT  teT 


so 


V = max  min  A(s,t)  = max  min  A(s,t) 
seS  teT  seS  teT 

< max  min  A(s,t)  = V 
seS  teT 


since  S C S. 

This  corollary  implies  that  the  first  ("outside")  player 
generally  does  better  with  an  adaptive  strategy,  but  the  second 
("inside")  player  cannot  Improve  his  payoff  by  employing  an 
adaptive  strategy. 
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Let  G = (A,S,T)  denote  the  nonadaptive  game  with  payoff 
A and  strategy  sets  S and  T.  This  adaptive  game  is  denoted 
by  6 = 

Corol 1 ary.  If  G has  a saddlepoint,  then  so  does  6. 

Proof.  Prom  the  above  corollary 

V £ V £ V <_  V . 

If  G has  a saddlepoint,  the  v = v,  so  V = V which  implies 
that  G has  a saddlepoint. 

This  corollary  was  also  established  by  Berkovitz^  in  the 
context  of  differential  games. 

The  following  example  shows  that  6 may  possess  a saddle- 
point while  G does  not.  In  Figure  1,  the  extensive  form  of 
6 is  given  with  terminal  payoffs  as  indicated.  The  normal 
form  of  the  game  is  given  in  Table  1,  where  we  employ  the 
notation 

(R|L,R) 

to  stand  for  the  strategy: 

go  down  and  to  the  right  when  in  period  1; 
go  down  and  to  the  left  when  in  period  2 if 

your  opponent  went  left  in  period  1,  otherwise, 
go  down  and  to  the  right  in  period  2. 


^Berkowltz,  L. , Lectures  on  Differential  Games,  in  Kuhn,  H.  and  Szego,  G. 
(editors)  Differential  Games  and  Related  Topics y North-Holland,  1971. 
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Figure  1.  THE  ADAPTIVE  GAME 
(EXTENSIVE  FORM) 


Table  1.  THE  ADAPTIVE  GAME--NORMAL  FORM 


— . 

. — ^ 



»« — > 

a: 

a: 

— i 

on 

—1 

on 

-J 

—1 

on 

-j 

on 

on 

-J 

— j 

-J 

on 

q: 

on 

o: 

•S ^ 

S ^ 

" — 

(L|L,L) 

0 

0 

0 

0 

1 

1 

2 

2 

(L|L,R) 

0 

0 

0 

0 

0 

0 

0 

0 

(L |R,L) 

2 

2 

2 

2 

© 

0) 

2 

2 

(L|R,R) 

2 

2 

2 

2 

0 

0 

0 

0 

(R|L,L) 

0 

0 

0 

0 

0 

0 

0 

0 

(R|L,R) 

0 

0 

0 

0 

0 

0 

0 

0 

(RlR.L) 

0 

0 

0 

0 

0 

0 

0 

0 

(R|R,R) 

0 

0 

0 

0 

0 

0 

0 

0 

There  are  2 saddlepoints  (circled). 
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In  Figure  2,  the  extensive  form  of  the  nonadaptive  game  is 
displayed.  The  corresponding  normal  form  is  given  in  Table  2. 
Note  that  there  is  no  saddlepoint. 


Figure  2.  THE  NONADAPTIVE  GAME  (EXTENSIVE  FORM) 
Table  2.  THE  NONADAPTIVE  GAME--NORMAL  FORM 


LL 

LR 

RL 

RR 


^ o:  ^ OH 

^ ^ OH  OH 

0 0 12 

2 2 0 0 

0 0 0 0 

0 0 0 0 


In  these  examples  we  have  v=  0,  V=v=V=V=l.  Note 
that  V = V,  a reflection  of  the  fact  that  one  of  player  II 's 
MinMax  adaptive  strategies  is  actually  a nonadaptive  strategy 
(R|L,L). 
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